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ABSTRACT 

This report describes an evaluation of Project 
Success, a dropout prevention program ac Crockett High School in 
Austin (Texas) . The class centered program provided incentives for 
improved educational performance and attendance of at-risk 
ninth-grade students in the 1990-91 school year. The project served 
103 students <wer the fall 1990 and spring 1991 semesters. The 
evaluation includes comparison of program participants with a similar 
group at another school* The evaluation producer, the following major 
findings: (1) participants had a dropout rate of 6.8 percent, which 
is lower than the comparison group; (2) the project did not have a 
positive impact on achievement because participants did net achieve 
predicted test score gains; (3) participants 1 attendance rates 
declined from the 19*9-90 to the 1990-91 school year, however, the 
extent of the decrease was less for the participants than for the 
comparison students; (4) it was not possible to determine the impact 
of the incentives on students 9 attendance and achievement; and (5) 
teachers perceived conference attendance, planning periods, and 
smaller class size as fringe benefits rather than performance 
incentives. Statistical data are provided in two figures. (JB) 
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Program Description 

^he Fund for the Improvement 
and Reform of Schools and 
Teaching(FIRST) grant project 
was a dropout prevention pro- 
gram at Crockett High School 
called Project Success. The pro- 
gram provided incentives for 
improved educational perform- 
ance and attendance of at-risk 
ninth-grade students. The pro- 
gram was classroom centered. 
Curriculum was adjusted to fit the 
special needs of the students. 
Some teachers provided extra 
tutoring for the students. Teach- 
ers involved in the project were 
given incentives including a com- 
mon collegia! planning period and 
paid school leave to attend con- 
ferences. The project served 103 
students over the fall, 1990 and 
spring, 1991 semesters. 

To assess the effectiveness of the 
program in terms of student 
performance, FIRST students 
were compared to similar stu- 
dents at Crockett and to a com- 
parison group at another District 
high school to see if the FIRST 
students' performance improved 
over time and/or in relation to a 
comparison group. Information 
on at-risk Crockett ninth graders 
for 1988-89 through 1990-91 is 
also included. 



Major Findings 

1. FIRST students had a dropout 
rate of 6.8%, lower than the 
dropout rate for other at-risk 
students at Crockett and the 
comparison group. Participa- 
tion in the FIRST project 
appears to have had a positive 
impact on student dropout 
rates (pages 4-5). 

2. The FIRST project did not 
have a positive impact on 
achievement because these 
students did not achieve 
predicted test score gains. 
FIRST students scored below 
predicted levels on all TAP 
subtests; however, they did 
gain at least one year on all 
subtests except Language 
(pages 6-7). 

3. FIRST students' attendance 
rates declined from the 1989- 
90 to the 1990-91 school 
year. However, the extent of 
the decrease was less for the 
FIRST students than for the 
comparison students (page 5). 
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4. Students were given incen- 
tives for achievement and at- 
tendance at the end of the 
fall, 1990 semester and at the 
end >f each six weeks during 
the spring, 1991 semester. 
Because of other aspects of 
the FIRST program, the 
impact of the incentives on 
students' attendance and 
achievement cannot be 
determined (pages 1-2). 

5. Teachers perceived confer- 
ence attendance, planning pe- 
riods, and smaller class size as 
fringe benefits rather than 
performance incentives 
(page 2). 
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Crockett's Project Success: 1990-91 Evaluation Report 

Final Report 



INTRODUCTION 

The Fund for the Improvement and Reform of Schools and Teaching (FIRST) grant project, 
"Restructuring for Student Success: A Dropout Prevention Model, 1 ' was a dropout prevention 
program at Crockett High School. Called Project Success at Crockett, the program was 
designed tc provide incentives for improved educational performance and attendance of at-risk 
students entering the ninth grade as well as for improved performance of teachers in the 
project. Teachers and administrators developed a plan for restructuring the educational 
program to better meet the needs of at-risk students. The plan was designed to involve 
teachers, parents, and administrators in the development and implementation of the program. 
Rather than fragmenting services by providing assistance outside the regular classroom or 
building, the program provided students with additional supportive services in the regular 
classroom. Incentives for students, such as material objects and special awards days, were 
provided both for improved attendance and academic achievement. Incentives for teachers, 
such as a common collegial planning period, paid school leave to attend conferences, funds 
for consultant fees for pertinent at-risk inservice tc^ics, funds to purchase supplementary 
teaching materials, awards, and gifts of travel, were also available. The project served 103 
students over the fall, 1990 am spring, 1991 semesters. 

Crockett High School committed the following staff: 

o Four full-time teachers, 

o Two vocational teachers, 

o Two part-time teacher aides, 

o A part- ne parent intervention specialist, 

o A grade level counselor, 

o A grade level assistant principal, and 

o A Communities in Schools social worker. 



INCENTIVES 

Student Incentives 

Students were given incentives both for attendance and achievement. During the fall, 1990 
semester students earning all A's were awarded a $100 savings bond. In addition, students 
earning all A's and B's received a $50 savings bond. Students also received a $50 savings 

1 
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bond for having perfect attendance. During the spring, 1991 semester these same incentives 
were awarded at the end of each six-week period. A total of 30 students received attendance 
and achievement awards in the spring semester. 

In addition to the larger awards, students were given incentives at the discretion of the 
individual teacher. These included: 

o Field trips, 
o Free time, 

o Sessions on basketball, cosmetology, and handbells, 

o Extra credit, and 

o Small gift certificates. 

Students were given these incentives on a schedule determined by each teacher. It was 
hoped that these incentives would motivate the students both to attend school and to 
achieve in their studies. Whether the incentives alone served this goal cannot be 
determined. Because the FIRST program offered a number of special services to the 
students, in the absence of a controlled study in which the various services were given or 
withheld from different groups, ; .t is not possible to determine which part or combination of 
parts had an impact on student performance. 

Teacher Incentives 

Plans were to offer teachers incentives based on the improved performance of their students. 
However, when interviewed, only two of six teachers indicated that they had received 
incentives. The conference attendance, planning periods, and smaller class size seemed 
to be perceived as fringe benefits rather than performance incentives. Perhaps in future 
years teacher incentives could be improved and a more direct link between student 
performance and teacher rewards established. However, none of the teachers expressed 
dissatisfaction with the current system. One indicated that the incentive was the improved 
performance of the students. 



COMPUTER USB 

The project purchased five computers and additional software to assist students in the four 
basic subject areas: language arts/reading, mathematics, science, and social studies. These 
new computers were set up in the regular classroom instead of a separate lab. Another lab 
provided by Communities in Schools was accessible during the entire school year for all 

tc ,rs. 
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Teachers were asked in interviews during February and March, 1991, to indicate how often 
students used the computers. Most (4) indicated either that there was no set schedule for 
student use or that software was not yet available. One teacher indicated that computers were 
used in the classroom two days a week for 20 minutes each day for a total of 40 minutes a 
week. 

CURRICULUM ADJUSTMENT 

Teachers, when interviewed, indicated a variety of ways the curriculum was adjusted to meet 
the needs of FIRST students. All indicated that they did not rely on the textbook or that they 
modified its use. Teachers also reported that students had different needs that the teachers 
tried to be aware of and meet. Teachers used real-world examples and fewer visual materials. 
One teacher offered tutoring before classes; another used the aide to help students who la/ged 
behind. The major theme in curriculum adjustment was listening to the needs of the students 
and giving them extra attention. 

PARENTAL INVOLVEMENT 

An effort was made to involve parents of FIRST students in the education of their children. 
Teachers and teacher aides called parents and related both positive and negative aspects of the 
FIRST students' academic performance and behavior. Parents were also contacted if students 
were absent. A "Back to School" night was held at the beginning of the year to acquaint 
parents with the program and introduce them to the teachers. At the end of each semester an 
awards assembly or banquet was held for FIRST students and their parents. Nearly all of the 
parents attended the banquet in the spring and contributed food. When interviewed, the six 
teachers indicated that parents were either somewhat (2) or a little bit (4) involved in the 
project. 

STUDENT PERFORMANCE ON MEASURES OF ACADEMIC SUCCESS 

In order to assess the effectiveness of the program in terms of student performance, FIRST 
students were compared to similar students at Crockett and at another District high school to 
see if the FIRST students' performance improved over time and/or in relation to a comparison 
group. The program had a dual goal of improving both the academic performance of the 
students and improving their attendance and dropout rates. According to staff interviewed, 
for the purposes of the program, improved attendance was defined in two ways. First, 
attendance was considered to have improved if higher attendance rates for FIRST students 
were observed compared to similar at-risk students. A second way of defining improved 
attendance was to compare rates for the 1990-91 school year to rates in 1989-90. 
Participation in FIRST was also hoped to raise student achievement. Staff indicated in 
interviews that, for the purposes of the program, improved achievement would be reflected by 
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FIRST students' performing better than similar students both on standardized tests and grade 
point averages. 

The performance of FIRST students was compared to at-risk ninth graders at another District 
high school. The school was selected for comparative purposes because it has similar 
enrollment size and demographic characteristics to Crockett. The two groups were not 
exactly the same demographically. The FIRST group was predominantly male (73%) while 
the comparison group is more equally divided between male (55%) and female (45%). The 
FIRST group was 51% Hispanic, 10% Black, and 39% Other, while the comparison group 
was 34% Hispanic, 37% Black, and 29% Other. Even with these differences a comparison 
between the two groups is valuable. The comparison group provides a comparative group of 
similar students that are not at Crockett High School. In addition, the school represents the 
best overall match in terms of school size, location, and demographics. 

FIRST student performance was also compared to other at-risk ninth graders at Crockett High 
School. The program did not include all Crockett ninth graders. Students were recommended 
for participation by their middle/junior high school counselors, their ninth grade teachers after 
the first few weeks of school, or because they were on the District at-risk list. This list 
contains students who are at-risk for a variety of reasons including: 

o Being overage for grade, 

o Scoring two or more years below grade level on a norm-referenced standardized 

achievement test, 
o Failing two or more courses during one semester, 

o Failing any section of the most recently administered state-mandated minimum skills 
test, or 

o A combination of the above categories. 

There were no set selection criteria, and students had the option of participating or not. This 
self-selection process may have had an impact on student performance. 

Baseline information is also included on at-risk Crockett ninth graders for the 1988-89 and 
1989-90 school years. This information is provided to demonstrate the demographic and 
performance characteristics of Crockett at-risk students before the implementation of FIRST. 

Dropouts and Attendance 

The dropout rate for at-risk Crockett ninth graders has been increasing since the 1988-89 
school year. In 1988-89, the dropout rate was 12.4%. In 1989-90, the dropout rate rose to 
13.4%, and in 1990-91, it rose again to 19% at the end of the sixth six weeks. FIRST 
students had a dropout rate of 6.8% at the end of the sixth six weeks. This is well 
below the rate of the population of at-risk ninth graders at Crockett The comparison 
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group's dropout rate was greater than the rates of both the FIRST students and Crockett at- 
risk students, at 22% at the end of the sixth six weeks. Participation in the FIRST project 
appears to have had a positive impact on student dropout rates. 

In terms of attendance, at-risk Crockett ninth graders have basically maintained the same rates 
from the 1989-90 to the 1990-91 school year. The fall, 1989 rate was 82%, spring, 1990 was 
78%, fall, 1990 was 82%, and spring, 1991 was 84%. FIRST students' attendance rates were 
greater than Crockett at-risk ninth graders in both the fall, 1990 (92%) and spring, 1991 
(90%) semesters. The comparison group attendance rates for fall, 1990 and spring, 1991 were 
78% and 76%, respectively. Attendance rates for both the FIRST students and the 
comparison group were lower in the 1990-91 school year than in 1989-90 school year. 
However, FIRST students had a 6% decline in attendance from 1989-90 to 1990-91, while the 
comparison group's decline in attendance was 9%. Attendance rates for all groups are shown 
in Figure 1. 

FIGURE 1 

ATTENDANCE RATES FOR FIRST STUDENTS AND COMPARISON GROUPS FOR 

THE 1989-90 AND 1990-91 SCHOOL YEARS 



1 

GROUP 


FALL 
1989 


SPRING 
1990 


FALL 
1990 


SPRING 
1991 


FIRST 


96% 


93% 


92% 


90% 


COMPARISON 


85% 


80% 


78% 


76% 



Findings with respect to FIRST students' attendance are mixed. FIRST students' 
attendance rat<* declined from the 1989-90 to the 1990-91 school year. FIRST students 
did have higher attendance rates for the 1990-91 school year than students in the comparison 
group, but the FIRST students had higher attendance rates in the 1989-90 school year, before 
their participation in the project. The attendance rates of both the FIRST students and 
comparison students declined from 1989-90 to 1990-91. However, the extent or the 
decrease was less for the FIRST students than for the comparison students. Therefore, 
the program may have had some effect in slowing the rate of decline in attendance among 
program students. 
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Achievement 

Student performance on standardized tests was exarrned as a measure of school success. 
Ninth-grade students in AISD take the Tests of Achievement and Proficiency (TAP). TAP 
consists of subtests in Reading Comprehension, Mathematics, Written Expression, Social 
Studies, Science, and Using Sources of Information. The Office of Research and Evaluation 
has designed a special set of computer programs called the Generic Evaluation System 
(GENESYS) to gather outcome information on programs of interest. Part of the GENESYS 
analyses is the Report on Program Effectiveness (ROPE). A regression analysis, ROPE is a 
statistical manipulation of test scores that yields a prediction of how the students should score 
on achievement tests based on students districtwide with similar characteristics, including the 
following: 

o Previous achievement level, 

o Age, 

o Sex, 

o Ethnicity, 

o Estimate of family income, 

o Whether or not the student received a free or reduced-price lunch, 

o Whether or not the student was reassigned, and 

o The average pupil-teacher ratio for the student's grade at his^e^ school. 

Some at-risk characteristics (such as failing two or more courses during one semester, or 
failing any section of the most recently administered state-mandated minimum skills test) are 
not part of the prediction in the ROPE analysis. 

A measure of program effectiveness is made by comparing the students' actual performance 
relative to the prediction. The tests used for the ROPE are Reading Comprehension, 
Mathematics Written Expression (Language), and Using Sources of Information (Work- 
Study). The ROPE analysis was performed on the FIRST students, the other at-risk ninth 
graders at Crockett, and the comparison students (see Figure 2). 

In the spring of 1991, FIRST students scored below the ninth-grade level on all tests. 
(Students would be expected to score at the 9.8 grade equivalent on the tests to be considered 
on grade level.) Students' scores were significantly less than predicted levels on all subtests 
except Work-Study. However, FIRST students' scores did improve by one grade level 
between 1989-90 and 1990-91 on all subtests except Language, an improvement paralleled by 
other at-risk ninth graders at Crockett. The comparison group did not score at the ninth-grade 
level on any subtests, and only on Work-Study did these students' scores improve by one 
grade level from 1989-90 to 1990-91. The comparison group's test scores were also 
significantly below predicted levels except on Work-Study (see Figure 2). 
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FIGURE 2 

FIRST STUDENTS' AND COMPARISON GROUP'S GRADE EQUIVALENT 
PERFORMANCE ON TAP 1989-90 AND 1990-91 

ROPE, SPRING 1990 TO SPRING 1991 MEAN GRADE EQUIVALENT 
FIRST CROCKETT COMPARISON 







AT-RISK 


READING COMPREHENSION 




111 


Number of Students 


75 


1990 Grade Equivalent 


6.0 


7.9 


1991 Grada Equivalent 


9.0 


8.8 


Gain 


1.0 


1.0 


Over/Under Predicted 


-.5 


-.4 


Program Effectiveness 






Range for 0 (+/•) 


0.4 


0.4 



0.5 



MATHEMATICS 

Number of Students 74 109 56 

1990 Grade Equivalent 8.1 7.8 7.4 

1991 Grade Equivalent 9.1 8.5 8.1 
Gain 1.0 0.8 0.8 
Over/Under Predicted -J 

Program Effectiveness 

Range for 0(*A) 0.4 0.3 0.4 



LANGUAGE 

Number of Students 72 115 55 

I960 Grade EqulveJent 8.2 6.0 7.9 

1991 Grade Equivalent 8.6 8.3 6.4 

Gain 0.4 0.3 0.5 

OverAJnder Predicted - 6 - 7 - 5 
Program Effectiveness 

Range for 0 (♦/-) 0.4 0.3 0.5 



WORK STUDY 

Number of Students 75 103 52 

1990 Grade Equivalent 6.1 7.8 7.2 

1991 Grade Equivalent 9.6 9.0 8.5 
Gain 15 12 1 0 3 
OverAJnder Predicted - 3 2 
Program Effectiveness 0 0 
Range for 0 (♦/•) 0.4 0.4 0.5 



Based on the ROPE analysis, the FIRST project did not have a positive impact because 
these students did not achieve predicted test score gains. The FIRST students scored 
below predicted levels on all TAP subtests; however, they did gain at least one year on 
ali subtests except Language. The comparison group did not gain one year on any subtest 
except Work-Study. 
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Grade point average 

Crockett at-risk students had grade point averages (GPA) above the passing level for both the 
1989-90 and 1990-91 school years. In both fall, 1989 and spring, 1990, these students had a 
GPA of 68%. For the 1990-91 school year, their GPA was 70% for both the fall and spring 
semesters, FIRST students' GPA was 78% in fall, 1990 and 76% in spring, 1991. The 
comparison group's GPA for ihe 1990-91 school year was 68% for both semesters. FIRST 
students had a higher GPA than both other at-risk Crockett ninth graders and similar 
students at the comparison school. Because grade point averages for the eighth-grade year 
for FIRST students are unknown, program impact on GPA is inconclusive. 

CONCLUSIONS 

The FIRST project seems to have met its goal of lowering the dropout rate of program 
participants. FIRST students had higher attendance rates than similar students in the 1990-91 
school year; however, they also had higher attendance rates for the 1989-90 school year, 
before program implementation. While FIRST student attendance rates declined from the 
1989-90 to the 1990-91 school year, the rate of decline was less than that of similar students. 
FIRST student achievement was below predicted levels on standardized achievement tests, but 
the FIRST students did have greater gains than similar s idents. The awarding of incentives 
may have impacted student performance. In interviews, teachers noted that the students liked 
the material rewards offered through the program. Students may have also benefited by the 
special attention and adjusted curriculum offered by the teachers in the program. 
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